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MEMO: 


TO: R. Larson 

FROM: R. Covelli 


DATE: July 6, 1970 

SUBJECT: Errors in the Computation of FORVEL and LATVEL 


Simulations at Grumman have indicated that there might be an error in the 
computation of the horizontal velocities displayed on the crosspointers during 
P66. The quantities in question, FORVEL and LATVEL, are computed once 
every 0. 25 seconds in the RIO interrupt. Due to the high frequency of com¬ 
putation, the computations are made using single precision arithmetic in 
order to reduce the computer workload. 

The velocity data is scaled in RIO so that the least significant bit has a value 
of approximately 0. 6 feet per second. Since the inputs are truncated, the 
initial error may be this large. This error can propogate through the com¬ 
putation and may be as large as 3. 6 feet per second in the output to the 
crosspointers. 

An edit program was written to perform the same computations that RIO 
performs using the full precision of the 360. With the same inputs, the 
difference between the MAC edit and the AGC were as large as 3. 4 feet per 
second. All of this error is due to the inaccuracies of the single precision 
computations. 


The way to reduce this error is to use double precision arithmetic to compute 
FORVEL and LATVEL. This has the drawback of requiring more execution 
time and more fixed memory storage. Depending on the degree of improve¬ 
ment desired, a partial double precision calculation could be made, in which 
only key terms are handled in double precision. This might reduce the error 

to about 1. 2 feet per second. For greater accuracy, full double precision 
would be needed. 



